Mosaicing and Restoration from Blurred Image Sequence Taken with Moving Camera

A wide-area image can be synthesized from an image sequence taken with a moving camera by using image mosaicing techniques. However, motion blur caused by the motion of the camera may significantly degrade the quality of the synthesized image. In this paper, we propose a new method for generating a deblurred mosaic from an image sequence that is degraded by motion blur under the condition that we do not have any information about the intrinsic and extrinsic parameters of the moving camera during input acquisition. In this method, we assume the objects in the scene can be classified into two regions in order to handle depth. In this paper, the displacement vectors of the features, which are computed using the KLT feature tracker on the consecutive frames, are classified into two regions. Here, the classified vectors provide a Point Spread Function (PSF) of the blurred image, and a homography between two consecutive frames for segmentation and mosaicing. Experimental results show that the Signal to Noise Ratio of the generated images can be significantly improved by our proposed method. S. Singh et al. (Eds.): ICAPR 2005, LNCS 3687, pp. 598–607, 2005. c © Springer-Verlag Berlin Heidelberg 2005 mounted on an airborne platform with GPS/INS measurements using a parallelperspective representation [13]. Since such image sequences are sometimes captured using a moving camera, the motion makes the captured images blurred. As motion blur due to camera motion may significantly degrade the image quality, a considerable amount of research has been dedicated to restore these images. Blurred images can be deblurred by using image deconvolution [5]. A general motion blur PSF can be recovered from various devices [1]. Motion blur correction from multiple images has recently been tried as well. Rav-Acha et.al. proposed a method for image deblurring from two images having motion blur in different direction[7]. Synthesizing a super-resolved image from multiple images is also an active research topic [3, 8, 10]. Motion deblurring has also been addressed in the context of temporal super-resolution [2]. In most of the research on image restoration, motion blur is considered shift invariant. However, in practice, the motion blur is shift variant because a 3D scene has multiple depths, so the applicability is limited when the scene can not be considered flat. In addition, as many image mosaicing approaches do not consider motion blur, the quality of the mosaic synthesized from a blurred sequence is degraded. In this paper, we propose a new method for generating a deblurred mosaic from a blurred image sequence captured by a moving camera, in which the intrinsic and motion parameters are unknown. In our proposed method, we combine methods for motion image deblurring and image mosaicing so that we can synthesize mosaic images without motion blur from image sequences taken with a moving camera. The proposed method is achieved by deblurring each frame of the input sequence and generating a mosaic image from the deblurred frames. The proposed method also takes into account multiple regions with different depth and blur by segmenting each frame based on displacement differences of tracked points, and the estimation of the homography and PSF for each region. The proposed method first tracks a number of feature points over the input image sequences using the KLT feature tracker. By assuming that the object scene can be represented by two layers of planar regions, the displacement vectors of the features on the consecutive frames are classified into foreground points and background points. For each region, the displacement vectors are averaged for estimating the PSF of the motion blur. By applying the Wiener filter [5] with the estimated PSF for each region, the input image sequence can be deblurred. Our method then merges all the images in the input image sequence by image mosaicing techniques. For the image mosaicing, the homography of each region between the consecutive frames is estimated from the displacement vectors within the region. After the image mosaicing, we can finally synthesize a deblurred wide-area image from the input image sequence. We also have conducted experiments with various scenes consisting of a foreground object and 1 The homography maps the projected point from one plane in a 3D scene to another plane. Mosaicing and Restoration from Blurred Image Sequence 599